Skip to content

Added Type-hinting to methods + Update to Numpy v2 #273

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
ReinierKoops wants to merge 63 commits into
base: main
Choose a base branch
from
Open

Added Type-hinting to methods + Update to Numpy v2 #273

ReinierKoops wants to merge 63 commits into from

Conversation

@ReinierKoops
Copy link
Contributor

@ReinierKoops ReinierKoops commented Mar 14, 2025 •
edited
Loading

This PR contains:

  • Type checking for all methods (add literal checks, re-evaluate some checks, similar methods have similar expected inputs and outputs - for example, plot should return fig)
  • Update the figure produced by running SHAPRFECV (it is a discrete variable - not continuous).
  • Added some missing default parameters to functions (that were now passed via kwargs, i.e. groups for fit_compute)
  • Added more comments & docstring to better explain what is going on. (TODO: Re-evaluate all docstrings before merging the PR)
  • Updated the shap calculation function. Now better supports multiclassification, with aggregation, weighing and class selection options.
  • Added TQDM progress bar + manage better the extensive logging of some other libraries. Integrate that better (+ fix: Log in docs is too long #274).
  • Made the unit tests more robust (remove some, updating others, added some, specific for classes - catboost, lgbm, xgboost, regression, classification, random forest, others, cv, multiclass, ...) (TODO: Merge fixtures which are similar, Make sure all fixtures are in conftest.py, merge tests which have overlapping functionalities that are tested, reduce the number of tests, reassess added assertions if they can be removed or merged, re-introduce tests from the old Probatus that test for specific bugs and edge cases that make sense to test)
  • Update to Numpy 2+ & Shap latest (and other package updates)
  • Backwards compatibility for Numpy 1.*
  • Update docs, notebooks: add examples of multiclass + regression (and fix: Log in docs is too long #274), other functionalities like different multiclass options, overview of the new plots, examples of legacy imports & newer imports, intricacies of the sample size; effect on speed and additivity check, notebook/doc on the changes in this new version
  • Use newer plots from SHAP (replace summary_plot by bar plot), add consistent styling to plots that look (more) modern
  • Integrate the use of Pipelines
  • Update readme to include all the latest changes
  • Remove deprecations, old dead code and try to fix warnings that can be fixed

@PaulZhutovsky

@ReinierKoops
fix
1747641
@ReinierKoops
improve plots (displays discrete stats instead of non-existent contin…
08c6e6f 
…uous)
@ReinierKoops
add more comments, cleanup some code, add type checking
732c88c
@ReinierKoops
fix utils
98c809d
@ReinierKoops
fix arrayfuncs
72d5133
@ReinierKoops
fix exceptions
87637b4
@ReinierKoops
fix model_interpret
49b3eb2
@ReinierKoops
fix test shap dependence
ab4c1f6
@ReinierKoops
fix test model interpret
167ed29
@ReinierKoops
fix arrayfuncs
40d97ea
@ReinierKoops
fix model_interpret
a8b3a53
@ReinierKoops
fix model_interpret
8b6a6cb
@ReinierKoops
fix some more files
0c8d064
@ReinierKoops
add more comments, cleanup some code, add type checking
7864142
@ReinierKoops
add more comments, cleanup some code, add type checking
98eaf5c
@ReinierKoops
add more comments, cleanup some code, add type checking
ebb7ad0
@ReinierKoops
add more comments, cleanup some code, add type checking
ef0c8d4
@ReinierKoops
add more comments, cleanup some code, add type checking
faa2bd2
@ReinierKoops
add more comments, cleanup some code, add type checking
bb80871
@ReinierKoops
add more comments, cleanup some code, add type checking
9bec0ea
@ReinierKoops ReinierKoops changed the title Simplify & Improve Probatus code Added Type-hinting to methods Mar 15, 2025
@ReinierKoops ReinierKoops changed the title Added Type-hinting to methods Added Type-hinting to methods + Update to Numpy v2 Mar 15, 2025
@ReinierKoops
Copy link
Contributor Author

ReinierKoops commented Mar 15, 2025

At the moment catboost does not work. This will be fixed in the newer version of Probatus. Expect catboost to update to numpy v2 any day.

@detrin
Copy link
Contributor

detrin commented Mar 17, 2025

If you have a spare time I would definitely give it a try. I normally use uv, but for debugging of dependencies, poetry is more suitable.

@ReinierKoops
Copy link
Contributor Author

ReinierKoops commented Apr 4, 2025 •
edited
Loading

Still waiting for the catboost release. can be any moment it seems. In the meantime performing some refactoring (making wrapper classes for estimators, datasets and shap to simplify the implementation and detangle logic)

@ReinierKoops
Copy link
Contributor Author

ReinierKoops commented Apr 8, 2025 •
edited
Loading

Started making separate plotting classes. This will allow standardization (style, output) of plots among the main classes (rfe, resemblance, interpretability) and also extend the possible plots that can be plotted from each of those classes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@ReinierKoops @detrin